Assessment of Vaccines

Slides: https://www.andreashandel.com/presentations/

2023-10-30

Vaccines are pretty good

xkcd.com

Evaluation of vaccines

How do we determine if vaccines are good?

  • Safety
  • Immunogenicity
  • Efficacy/Effectiveness
  • Cost-effectiveness

Vaccine Development

Knipe et al Science 2020

Outcomes of interest

Vaccines (partially) protect those who receive them (direct/individual effect):

  • Reduction in risk of infection/symptoms/hospitalization/death.
  • Reduction in strength of symptoms.

Outcomes of interest

Vaccines (partially) protect those who receive them (direct/individual effect):

  • Reduction in risk of infection/symptoms/hospitalization/death.
  • Reduction in strength of symptoms.

Vaccines can also protect non-vaccinated contacts (indirect effect).

  • Reduction of susceptibles in the population leads to overall reduced spread (contagion effect).
  • Reduction of infectiousness/transmission potential leads to reduced spread (infectiousness effect).

See Halloran & Hudgens 2016 CER and references therein.

Indirect effect example

  • Vaccine 1 reduces risk of clinical infection by 70%, reduces infectiousness by 30%.
  • Vaccine 2 reduces risk of clinical infection by 30%, reduces infectiousness by 70%.

Gallagher et al, medRxiv 2020

Ways to evaluate vaccine impact

Measure it:

  • Challenge studies
  • Clinical trials
  • Observational studies

Estimate it:

  • Correlates of Protection

Measuring vaccine impact

Challenge studies

  • One group receives the vaccine, the other placebo.
  • Both groups are challenged with the pathogen under consideration.
  • Measures vaccine efficacy (VE).
  • Well-controlled, can use small(ish) sample size.
  • Somewhat unrealistic (e.g., high challenge doses).
  • Direct effect only.
  • Sometimes not feasible/ethical.

xkcd.com

Clinical trials

  • One group receives the vaccine, the other placebo.
  • Groups are followed and outcome (infection/disease/etc.) recorded.
  • Measures vaccine efficacy (VE).
  • Good balance between controlled and real-world setting.
  • Usually needed for FDA approval.
  • Only works if infections are high (not good for emerging pathogens).
  • Can measure direct and indirect effects (but usually only direct).
  • Expensive.

Polack et al 2020 NEJM

Observational studies

  • Taking vaccine is up to individuals (so must be licensed).
  • Cohort and case-control (e.g., test-negative) design.
  • Measures vaccine effectiveness (VE).
  • Most “real”, least controlled.
  • Can lead to biased estimates.
  • Can measure direct and indirect effects.
  • Can be fairly inexpensive.

xkcd.com

Test-negative design

Sullivan et al 2014 Exp Rev Vac

Measuring vaccine impact - summary

  • Different study designs are available/useful.
  • As you learned from the Halloran & Hudgens paper, because of dependent happenings, designing and analyzing vaccine studies to properly capture all vaccine effects can be tricky.
  • Generally, a “classical” convincing phase 3 clinical trial is required to get approval (but see e.g., Ebola vaccine, H5N1 influenza vaccine).
  • Measuring actual outcomes is always expensive and time-consuming, sometimes not feasible (e.g., SARS-CoV-3 or H5N1 influenza vaccines).

Estimating vaccine impact

xkcd.com

Correlates of protection (CoP)

  • Determining an immunological quantity that correlates with protection can make vaccine assessment easier.
  • Finding correlates of protection (for vaccines) is very valuable (but can be tricky).

xkcd.com

Vaccine CoP terminology

  • It’s a mess.
  • Some individuals (e.g., Plotkin) mean by “correlate” a mechanistic/causal entity.
  • Some individuals mean by “correlate” something that correlates and might or might not be mechanistic/causal.
  • Other terms are used by some to try to be clearer (e.g., surrogate, mechanistic CoP). See e.g. Plotkin & Gilbert 2012 CID.
  • Protection is also often not clearly defined.

Correlates of protection

  • An “absolute correlate” (a la Plotkin) does not exist.
  • Levels at which a CoP leads to protection depend on pathogen, host, outcome, etc.

CoP - SARS-CoV-2 Example

Khoury et al 2021 Nat Med

CoP - SARS-CoV-2 Example

Khoury et al 2021 Nat Med

CoP - Influenza Example

Coudeville et al 2010 BMC MRM

CoP - Influenza Example

CoP - Influenza Example

CoP - Influenza Example

CoP - Influenza Example

CoP - Influenza Example

CoP - Influenza Example

Age Group 50-64

CoP - Influenza Example

Age Group 65+

CoP - Influenza Example

Age Group 18-49

Estimating vaccine impact - summary

  • Using CoP can speed up approval process.
  • CoP can depend on details of vaccine (e.g., LAIV vs. IIV) and hosts (e.g., children vs. adults).
  • A full mechanistic understanding of
    vaccine -> immune response -> protection
    is still lacking for any vaccine (afaik).

Keep going?

Image by Aline Dassel/Pixabay

Research project - Assessing influenza vaccine candidates

Current Influenza vaccines

  • Need to be reformulated almost every year because of virus evolution
  • Need to be taken annually, due to virus evolution and vaccine waning
  • Are not very good, even if the vaccine and circulating strains match

Future Influenza vaccines

  • Should protect for a long time (lifelong?)
  • Should have high efficacy
  • Should protect against a wide range of strains

Universal flu vaccine challenges

  • Many
    • How to assess/compare vaccine candidates

How do we define a vaccine response?

Quantifying vaccine responses

Quantifying vaccine responses

Quantifying vaccine responses

  • Magnitude: \(\frac{1}{N}\sum_{n} log(\textrm{TI}_{n,j=1})\)
  • Overall strength: \(\frac{1}{N*J}\sum_{n} \sum_{j} log(\textrm{TI}_{n,j})\)
  • Breadth: \(\frac{1}{N*J}\sum_{n} \sum_{j} \textrm{SC}_{n,j}\)

SC = Seroconversion, TI = Titer Increase (D28/D0), n = individuals, j = Strains.

Comparing vaccine responses

A new method to quantify/compare vaccine responses

  • Organize strains by antigenic distance
  • Fit a model to more robustly estimate magnitude/breadth/strength

Strain distance

Strain distance measures

  • Time: absolute difference in years of strain isolation.
  • Sequence: Some measure based on sequence difference.
  • Biophysical: Measures based on computed or measured biophysical properties.
  • Phenotypic: Antigenic cartography based on HAI assays.

“Our” data

  • Data from UGAFluVac study
  • Individuals received vaccine, response to multiple strains was tested

Strain distance measures

Strain distance measures

Quantifying vaccine responses

Comparing vaccine responses

Testing our method

We sampled from the panel of heterologous strains from UGAFluVac to mimic different labs

Testing our method - the results

The table shows the coefficient of variation for each outcome.

Current method Proposed method
Magnitude 0.088 0.103
Breadth 0.059 0.431
Overall strength 0.083 0.081

Our new method is worse (more variable)!

Testing our method with simulations

  • Create a universe of 50 possible heterologous strains with varying antigenic distances.

  • Create 10 lab panels by randomly sampling 9 strains and adding the homologous strain (distance of 0).

  • For each lab, generate 100 random individuals by simulating flu vaccine response titers from a model that shows linearly reduced response with increasing antigenic distance.

Simulation results

Current method Proposed method
Magnitude 0.025 0.008
Breadth 0.199 0.020
Overall strength 0.155 0.007

Now our new method is better. Hm…

The culprit

Simulation results with 30% censored data

Current method Proposed method
Magnitude 0.028 0.033
Breadth 0.290 0.316
Overall strength 0.137 0.071

With censored data, the current method looks artificially good.

Research Project Summary

  • Our proposed new method seems to be generally more robust.
  • If a good amount of censored data are present, the current method falsely under-estimates the uncertainty.
  • Our method also doesn’t properly handle the censored data (yet).
  • We need to update our method to properly deal with the censored values. Then we can do another comparison of our method and the current approach.

Wrap-up

  • Phase 3 trials are still the gold standard.
  • There is increased recognition that indirect effects can be important and should impact decision making.
  • Finding better CoP for any vaccine continues to be an important area of research.

Questions?

https://phdcomics.com/

  • Slides: https://www.andreashandel.com/presentations/
  • Contact: https://www.andreashandel.com